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21sTCENTURYRENAISSANCE 


Six centuries ago a group of Italian inventors discovered 
how to portray 3D objects on 2D surfaces. Perspective 
had been known for centuries, but it took a multidiscipli- 
nary team to make it a successful communications me- 
dium. Not a team of artists, scientists and engineers, but 
a team of artist-scientist-engineers. Brunu Brunelleschi, 
Leon Battiste Alberti and Leonardo da Vinci were arche- 
types of the Renaissance: people who cultivated multiple 
intelligences to solve thorny problems. 


Renaissance painting gave birth to technical drawing, 
photography, cinema, television and most electro-optical 
media. For 600 years flat panel displays dominated global 
telecommunications, but they are giving way to VR and 
other immersive media. Can 21st century specialists 


invent new paradigms that have the powerful elegance of 
Renaissance engineering? 


At King’s College London, we think the future lies in a 
combination of science, technology and aesthetics. For 
one year our Connected Culture project recruited inter- 
disciplinary thinkers and doers to lay the foundations for 
21st century communications. With support from 9 in- 
stitutional partners, over 50 participants created 6 events 
that probed the expressive potential of 5G networks, 
spatial soundscapes and 3D video. 


Our results demonstrate how arts and technology can in- 
teract more fruitfully, and they also show how the univer- 
sity can use its naturally broad resource base to stimulate 
the broader ecosystem. For individuals we demonstrate 


the value of cross-training in solving problems that pay 
no heed to disciplinary boundaries. 


The massive challenges of this century require a holistic 
approach - compartmentalised thinking has brought un- 
foreseen and unwelcome results. We may do well to start 
a 21st century Renaissance that combines the rigour of 
science with sensitivity to both the human condition and 
the beauty of the natural world that surrounds it. 





















To start the project, networking professionals 
worked with dramatists to develop Fundamen- 
tal, a series of short plays intended for black box 
theatres. 


Fundamental casts a modern light on relation- 
ships. Taking inspiration from recent science, 
its creators craft stories that resonate with the 
uncanny universe of quantum physics. 


Connected Culture required simple artistic con- 
cepts to test complex engineering propositions. 
The team extracted dramatic gestures from theo- 
ries of uncertainty and entanglement. These took 
form in the story of a woman reaching to herself 
across space and time. 


WORKSHOP2ANALOG 






We invented the ‘analog’ workshop to perform dry 
runs without building expensive systems. Proto- 
types were developed with cardboard, tape and 
coloured string that could be quickly tested and 
reconfigured. 


Producers from Battersea Art Centre coached 
the project team in ‘scratch’, an agile approach 
that encourages flat teams and co-creation ina 
non-linear working environment. 





By end of day, the team had a working technical 
configuration that saved considerable expense 
when it came to actually building the telecommu- 
nications that bring networked theatres to life. 





WORKSHOP3SSOUND 


During this workshop, we explored how to acous- 
tically bind three separate spaces. Today’s media 
sets a high bar for theatre technology. Somehow 
you must create a performance that surpasses 
the home experience. 


The team adopted the Soundscapes spatialised 
audio system invented at King’s College London. 
With lightweight software and speaker arrays, 
they were able to crisply place an actor’s voice at 
a specific spot on stage. 


Placement of sounds turned out to be a critical 
factor in psychological immersion. With Sound- 
scapes remote voices sounded immediate and 
spontaneous - they recaptured the excitement of 
live theatre. 
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At RADA we experimented with a rehearsal of 
A Little Night Music directed by Edward Kemp, 
who is also executive director of RADA. 


Our networking and audio processing technol- 
ogies were applied to two issues. The first was 
logistical. The performance took place in the Jer- 
wood Vanbrugh Theatre on RADA campus. On 
stage were actors and a piano played by musical 
director Michael Haslam. But instrumentalists 
from the Royal Academy of Music and an addi- 
tional pianist were in a remote studio. 


We modified RADA’s gigabit Ethernet to operate 
with 5G methodologies. Both spaces were served 
with studio microphone arrays and two-way video 
accessed by monitors that were hung over the 
pianos. The Soundscape system was set up in the 
Vanbrugh mainstage, and the remote studio had 
2x2 stereo. 


Soundscape spatialisation literally added a new 
dimension to networked performance. Because 
their output was placed in a specfic place, musi- 
cians playing in the remote studio felt present, 
and the workshop had fluid moments when the 
directors, performers and audience operated in a 
single physical-virtual space. 


WORKSHOPSNETWORK 





Photographs by Richard Eaton 


At Sadler’s Wells we partnered with Guerilla Sci- 
ence and international Hip Hop Festival Breakin’ 
Convention to present a ‘transatlantic cypher’ 
based on Epiphany, a multimedia artwork devel- 
oped by Ali Hossaini. A cypher is a freestyle vocal 
performance using rhyme and rhythm. 


In an engineering first, we used a mobile network 
to connect Sadler’s Wells with Plush Studios in 
New York City. A choir in New York, classically 
trained but singing blues, joined UK Beatbox 
Champions to perform themes from Epiphany. 


For audio we implemented 2x2x2 stereo. Multi- 
ple monitors carried video from New York to the 
audience at Sadler’s Wells, and the event was 
webcast by Guerilla Science on Facebook Live. 


Sound from New York was crisp with a broad fre- 
quency range, and well-executed pans in the mix 
created a feeling of presence, particularly within 
audience seating in front of stage. Video was de- 
layed 5 — 7 seconds, but audience members who 
were surveyed claimed not to notice. By contrast, 
delays in audio were painfully obvious. 





WORKSHOPCIMMERSION 



























The installation of Ouroboros at Click Festival was 
the final workshop. Held in Helsinger next to Hamlet’s 
castle every May, Click is Denmark’s premiere event 
for art and science. Ouroboros is a video cycle by Ali 
Hossaini that presents the history of the universe as 
an immersive 3D installation. 


Video was projected on three 10m x 3m walls that 
formed a square with the fourth side open to the fes- 
tival. Soundscape was implemented by ten speakers 
placed outside the installation. The carpeted interior 
contained 150 cushions for floor seating. Visitors were 
offered 3D glasses at the entrance. 


Ouroboros audio is an original ambient composition 
by Keir Vine. It draws on global music traditions and, 
thanks to Soundscape, offers a structured auditory 
space that fully complements the video. 


During its course, the audio describes a circle, a 
square and then a triangle. A particular musical form 
expresses each figure. For instance, the circle places 
visitors inside a Tibetan singing bowl. 


A visually impaired child enjoyed a full run of audio 
before describing its spatial structure with accuracy. 


CONCLUSIONS 


Immersion - not networking - is the creative 
and business driver for new generations of 
live performance. 


Networks are critical enablers for contemporary 
production, but artists and audiences are moti- 
vated by direct experience. 


From the margins to immersion 


For thirty years networked performance has lin- 
gered at the margins of theatre. Part of the rea- 
son lies in infrastructure: it 1s no secret that high 
bandwidth, low latency connectivity is essential 
to live collaboration. 5G promises to overcome 
past difficulties with superfast wireless service. 


Although a full 5G environment does not exist, 
we applied 5G methodologies to simulate its per- 
formance. We also simplified our creative impuls- 
es. Earlier efforts to carry a narrative resulted 

in complex failures that revealed no clear path 

to success. Instead we focused on gestures — sin- 
gular expressive concepts — that could be used to 
test engineering propositions. 


Other obstacles to earlier efforts related to work- 
ing methods. Traditionally trained artists and 
scientists have different approaches to creativity 
— something we decided to overcome. To main- 
tain an experimental posture, each event was 
treated as a workshop. Each workshop isolated a 
problematic element of networked performance. 
Each element required an artistically engineered 
solution. By exploring single topics in depth with 
ample cross-disciplinary support, we were able to 
discern conceptual, aesthetic and technological 
issues with networked performance while sug- 
gesting pathways to impact. 


Researchers who are cross-trained in art, 
science & humanities will power future 
progress. 


Our conclusion about immersion is curious when 
we consider high drama on networks: from the 
first telephone romance, now lost to history, to 
the pulsing cacaphony of social media. Drama 

is no stranger to distance, but, for the stage, it 
occurs despite, not because, of it. This counters 
our premise that better networks would support 
new generations of ‘networked’ stagecraft. Our 
investigation of audio clued us to what is at stake 
psychologically, and thus artistically, when per- 
formers interact at a distance. 


In previous networked performances that we 
observed, audio was carried on mono or stereo 
channels placed near video panels. Our approach 
required us to consider audio in isolation from 
video. As we conducted our first tests with Fun- 
damental, a series of plays inspired by physics, 

a major problem yawned. Conventional audio 
created a sense of dull disembodiment that 
undermined the drama. Fortunately we had the 
opportunity to incorporate Soundscape into later 
workshops. 


Soundscape is a software application for creating 
spatialised audio in two parameters: placement of 
individual sounds at a precise location, and repro- 
duction of particular sound environments. Unlike 
competing systems it does not generate a physical 
sound field model. Instead it uses signal process- 
ing to create psychoacoustic images. Because 
these 'sound images' reproduce the psychological, 
not physical, characteristics of audio waves, they 
require minimal processing, and they allow real 


Audio is as important as video in the 
creation of virtual spaces. 


time manipulation with inexpensive equipment. 


A Soundscape set up consists of five+ speak- 

ers arrayed in a circle that defines a stage. For 
Fundamental three stages were set, each with an 
actor at a different life phase: present, past and 
future. The dramatic gesture was introspection. 
(Or self as other.) Superficially it seems we could 
have have used a single speaker to represent re- 
mote personae. In practice, this felt like a person 
on a loudspeaker. 


We found that only spatialisation created a 
transparent feeling of presence. Psychological 
immersion requires a sense of space to contour 
the audio source, and Soundscape structured the 
audio field — and restructured the acoustic space 
— to create natural sensations. 


Auditory perception is attuned to qualities such 
as the shape of a room, its volume and the tex- 
ture of walls. These liminal traits inform the 
experience of presence, and more fundamentally, 
the sense of reality that pervades a sound repro- 
duction. At the beginning of the workshop, we 
assumed that control over direction and audito- 
ry volume was sufficient for telepresence. Our 
experiments revealed additional parameters that 
proved vital to the sense of drama. Creating a 
virtual space, or merging remote spaces into a 
unified experiential field, requires precise con- 
trol over perceptions of resonance, reflection and 
spatial volume. 


Workshops at RADA (Royal Academy of Dramatic 


CONCLUSIONSCONT 


Art), Sadler’s Wells and Click, a national festi- 
val of art and science in Denmark, underscored 
the centrality of immersion as a creative and 
technological driver. They also clarified the tight 
coupling between networks and edge processing 
applications for audio and video. 


An engineering first 

The networked performance at Sadler’s Wells / 
London and Plush Studios / New York was the 
first ever to happen over a mobile network. Dr 
Alain Renaud of MintLab, Prof Alex Carét of An- 
halt University and Drs Maria Lema Rosas and 
Fragiskos Sardis of King’s College London devel- 
oped workflow that enabled musicians to collabo- 
rate in real time across the Atlantic. 


Innovative methodology 


Connected Culture laid the foundation for sub- 
stantial changes in how the arts and technology 
interact; and also how King’s College London 
interacts and creates impact in the wider creative 
and commercial ecosystem. 


First it created a viable platform for innovation 
where the most important stakeholders, i.e. large 
corporates, SMEs/start-ups, stakeholders and 
universities, can co-create without major finan- 
cial or regulatory pressure. This is important as 
interaction between major players (e.g., Erics- 
son and the National Theatre) often occurs only 
at the procurement stage where it is too late to 
articulate needs and deliver well-fitted solutions. 
An innovative but neutral ‘sandpit’ environment 
is paramount in ensuring the best innovation 
return on every pound. 


This, in turn, underpins another fundamental 


shift from an ecosystem which judges incurred 
cost rather than total value. As an example, 
theaters mainly see connectivity as a cost item 
and not as an enabler for artistic progress. The 
joint working methodology has helped organ- 
ically nurture in-house innovation and thus 
shift perception from cost to value. For example, 
RADA now understands how 5G ultra low-laten- 
cy networking technology could underpin a new 
business model where theatres create a virtual 
consortium. 


Working methodology for engineers at King’s Col- 
lege London changed substantially. Notably, the 
iterative design methodology used in the project 
has made engineering innovation cycles more 
effective and efficient. 


Finally the tight co-creation process has enabled 
King’s College London to streamline impact and 
create a porous engagement model with the wider 
stakeholder ecosystem. Frequent field trips mo- 
tivated staff as innovative engineering concepts 
moved quickly from theory to practice. It has cre- 
ated new skills that are urgently needed by both 
the arts and engineering communities. 


In summary, the arts has triggered innovation 
in science and engineering, and science and 
engineering have shifted perception from cost to 
value. Scaling this working methodology to more 
verticals at a national level presents an impor- 
tant opportunity for the UK in coming years. 


Accelerated development 


Soundscape was invented by Prof Zoran Cvet- 
kovic and his team, Drs Enzo De Sena and Hu- 
seyin Hacihabiboglu in the Department of Infor- 


matics. The team confirmed that participation 
in Connected Culture accelerated Soundscape 
development. In 6 months the software climbed 
from a Technology Readiness Level of 4 to 7. (On 
a scale of 10.) 


The following areas benefited: 


Usability: A new user interface was developed for 
artists that includes automation, intuitive con- 
trols for placing sound sources and other func- 
tionality. 


Deployment: Testing versions to advanced proto- 
type in different environments proved the audito- 
ry perspective is robust and offered a framework 
for future development. 


Optimisation: Deployment of the software in real 
world environments enabled developers to test for 
bottlenecks and implement more efficient use of 
computational resources. The need for high fidel- 
ity sound during the artistic performances forced 
developers to achieve a step-change reduction of 
audible artefacts. 


Commercialisation: Interaction with artists in- 
formed the next steps in the development of the 
UI for commercial applications. Integration with 
JackTrip, Soundjack, Adobe Audition and other 
applications led to specification for development 
of plug-ins and interface with other software in 
creative workflow. 
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Rowan Pulford, Technical Project Manager 
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Royal Academy of Dramatic Arts 
Edward Kemp - Executive Director 
Michael Haslam - Music Director 
Faoileann Cunningham - actor 
Katie Buchholz - actor 
Christopher White - actor 

Liam Jefford - actor 

Eleanor de Rohan - actor 

Ralph Davis - actor 

Aimee Lou Wood - actor 

Lettice Thomas - actor 

Alfie Webster - actor 

Catherine Dryden - actor 


Royal Academy of Music 
Jason Carr - pianist 
Camilla Marchant - flute 
Stephen Payne - horn 
Ellen Roberts - double bass 


Room One 
Mahdi Yahya - founder, writer, director 
Laurance Chater - filmmaker 


Battersea Arts Centre 

Massie Wilson, Project Coordinator for Artistic Direc- 
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Meghan Peterson, Project Manager for Research & 
Higher Education 


Guerilla Science 

Pigalle Tavakkoli - Head of Experience 

Marisa Chazan - producer 

Kyle KyleMarian Viterbo - social media manager 
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Creative advisor 
Ben Cooper-Melchior 


Composer & sound designer 
Keir Vine 


Relative Motion 

Josh McNorton - producer 

Jennifer Tang - director 

Lulu Raczka - writer 

Aline David - movement director 
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Helen Coyston - production designer 
Aaron Dootson - lighting designer 
Max Taylor - audio technician 


Independent actors | London 
Gemma Brockiss 
Janine Harouni 
Tom Bailey 
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Gemma Brockis 
Claire Cordier 
Susie Trayling 
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Beatboxers | London 


Grace Savage 
MC Zani 
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Connected Culture was a collaboration between King’s 
College London's Department of Informatics and Dr Ali 
Hossaini, brokered and supported by the Cultural Institute at 
King’s College London. 


